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An effective website continuously improves to meet its users’ needs, constantly measures progress, 
and implements incremental changes toward desired goals. In the summer of 2020, the client 
requested that the Region 5 Comprehensive Center develop a plan to continuously improve a new 
website. On this website, the client teachers can upload, download, comment upon, and share 
instructional materials with other teachers. 


In this continuous improvement plan, we describe how feedback from teachers can improve the 
website’s efficiency and advance the client’s goals for a new initiative. As a result of this plan, the 
client will be able to identify continuous, incremental changes to make to the website based upon 
site analytics, performance metrics, and direct feedback from teachers. We designed this plan to be 
implemented by a single individual (i.e., the client staff member running the website) who may not 
have access to additional staff or funding. 


Below, we describe four simple steps of continuous improvement for a website: 1) identifying goals 
for the website, 2) identifying sources of feedback, 3) regularly obtaining feedback, and 4) using the 
feedback in meaningful ways. 
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1. Identifying Goals for the Website 


Before attempting to improve any website, it is important to identify your goals. If the website was 
spectacularly successful, what would that look like? After defining success, you can work backward 
to determine what must occur to achieve this end. We suggest taking some time to brainstorm, 
either independently or with a small group, about what success would look like. 


Once you establish what a successful website looks like, create a list of indicators of success. Here, 
an “indicator” just means something concrete that you can measure, count, or examine. For 
example, if a successful site has a robust training program for teachers, what observable effects 
would indicate the achievement of this goal? If the training program was successful, you might 
expect to observe teachers uploading fewer instructional resources in violation of copyright laws 
and a reduced need for staff to follow up on quality issues. Both of these effects are tangible and can 
be measured, representing indicators of success. 


Finally, prioritize your list of indicators by their level of importance and the degree of disruption or 
inconvenience resulting when the indicator is absent. 


Step 1 Example: 


In a brainstorming session, a client representative described success in this way: 
A successful website would have 
1. Open-author resources developed from teachers’ collaboration; 


2. Links to or uploaded resources which are created, customized, remixed, and viewed, as well 
as easily accessible and clear help materials; 


3. State educational institutions sharing links and open-author resources on the website; and 


4. Arobust teacher training program that provides a clear understanding of the process and 
potential issues regarding open-author resources. 


Of course, your priorities will change over time, and you can repeat this step as often as you need. 
One of the benefits of continuous improvement is the flexibility it allows in prioritizing and deciding 
what to tackle next. 
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Example from the client: 


During a brainstorming session, a representative identified the list of indicators below. Each 
indicator related to a website goal (see box above). After generating this list, we talked through 


the importance of each indicator and ranked them in rough order. 


Looking at this list, the representative agreed that the first and second indicators represented 
the highest priorities to the site. However, the greatest disruption in day-to-day efficiency was 
the third indicator: when teachers do not understand the instructions for using the site and 
upload materials incorrectly, fixing these problems can require significant staff time. Because of 
this, the staff member determined that the third indicator (“easily accessible and clear help 
materials”) was a high priority for continuous improvement. 


Example indicators listed by priority (from the client) 


Increased number of open-author resources are created, customized, remixed, and viewed. 
Increased number of resources linked to or uploaded and viewed. 
Easily accessible and clear help materials. 


Increased number of state educational institutions that share links and open-author 
resources. 


Robust teacher training program that provide clearer understanding, which trainers can 
use to anticipate potential issues. 


Increased number of resources developed from collaboration, whether that collaboration 
occurs on this site or in another venue. 
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2. Identifying Sources of Feedback 


Once you know what indicators you are prioritizing, develop one or two research questions that 
relate to your priority indicators. For example, if a priority indicator is “Easily accessible and clear 
help materials,” you might ask: “Can users easily find clear information in the website’s help files?” 


At this point, you have several different options for answering your research questions. Below is a 
list of common ways to collect relevant feedback: 


» Your website’s data. Most websites collect information on how many users visit the site, which 
pages they visit, if they downloaded anything, and where they exit the site (e.g., Google 
Analytics). Looking through the data you already have, are there any ways to answer your 
research questions? 


» Quick surveys. A quick survey of your users can tell you how users think or feel about 
something. Ideally, you can embed a quick one-item survey as a pop-up on a particular page of 
interest (e.g., “How useful did you find this information?” on a help page). However, if embedded 
pop-up surveys are unavailable, you can also email a survey to your users. In your email, state 
why you are asking the participants to complete the survey, why it is important, and how long it 
should take. 


» Peer conversations. Having a quick conversation with colleagues or peers at other 
organizations can be another helpful way to gain new perspectives. Simply showing your 
colleagues your research questions and the website can generate fruitful ideas for continuous 
improvement. 


» Usability tests. One of the best ways to gain feedback on a website is to simply talk to the 
website users directly. Ideally, this would come in the form of a one-on-one conversation with a 
user using a script (e.g., a usability test). Instead of asking the user questions about the website, 
it is best to provide them with a series of simple tasks that relate to your research questions. 
Then watch the user perform the tasks (either in person or remotely by sharing your screen) 
and ask the user to think aloud while performing the tasks. See Appendix A for more information 
about how to do usability tests. 
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3. Regularly Obtaining Feedback 


Too often, data collection and gathering feedback can feel overwhelming. When one is pulled in 
many directions (and particularly if one is managing a large website solo), the prospect of gathering 
information for continuous improvement can seem daunting. 


Who should gather the data? 


Ideally, the staff member running the website will collect feedback. However, if the staff 
member has concerns that their position might place unintentional pressure on respondents to 
reply positively, there are other options. To reduce these concerns, consider asking someone 
else to gather this information, such as 


» A graduate student looking for a project; 


» An intern interested in learning more about website development; or 


» Another staff member unconnected to the site (perhaps in exchange for assistance with their 
projects). 


To lighten the load, continuous improvement experts recommend that website owners pick a half- 
day once a month and dedicate that time to getting feedback. You could spend that morning talking 
to website users. Alternatively, you could pull together a quick survey and send it out. Whatever 
you choose to do, making regular feedback part of your monthly routine will ensure that data 
collection happens and is used regularly. 


Step 2 Example: 


To explore the research question “Can users easily find clear information in the website’s help 
files?” you might ask what percentage of website visitors access the help pages and how many 
open specific help files? Although this would not tell you if visitors found the help pages and 
files to be clear, you would learn how many visitors found them. 


Usability tests are another source of information. Such tests are appropriate for research 
questions that ask about respondents’ thoughts or feelings and examine respondents’ behavior 


when faced with a scenario. In this case, the research question asks whether respondents 


consider information to be clear and easy-to-find. Because both of these are opinions related to 
respondents’ inner feelings, a usability test is appropriate. 
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4. Using the Feedback in Meaningful Ways 


As you collect feedback, you will identify recommendations for improvement. Keep all 


recommendations in a single place. This gives you two advantages: 


» Youcan group similar recommendations together, regardless of their source. For example, you 
can group all of the recommendations related to a single page on the website into one category. 
Then when you are ready to work on that page, you can easily access the recommended changes. 


» Youcan see all of the various recommendations in a single place and prioritize them. We 
recommend that you rank all of your recommendations by 1) how impactful they would be if 
you fixed them, and 2) whether you can fix them. 


As you gather more feedback and make tweaks to the website your list of recommendations will 
change, and so will your priorities. It is important to review your list at least annually to confirm 


and identify your highest priorities. 


To make sure feedback is not collected 
Less is more and then left unused, it’s important to 
also regularly schedule some time to 


“Resist the impulse to add things. When it’s obvious in 
testing that users aren’t getting something, the team’s 
first reaction is usually to add something like an 


tweak or update the site based upon 
what you learn: once a month, for 


explanation or some instructions. But very often the example. The nature of these tweaks 
right solution is to take something (or somethings) will depend on the nature of the 
away that are obscuring the meaning, rather than feedback. As a principle, it’s a good idea 
adding yet another distraction.” to continuously make small 

~ Don’t Make Me Think | adjustments, rather than letting the list 


of fixes pile up. 


Conclusion 


Continuous improvement does not need to be overly difficult. Instead, the process should illuminate 
obvious issues and their fixes. By following the steps outlined in this report, you will be able to 
continuously improve your website and ensure that it stays current, meeting users’ evolving needs. 
When you succeed in attaining your website’s goals, your data collection efforts will also provide 
you with valuable quotes and data points illustrating your success. 
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Appendix A. How to Do a Usability Test 


Usability tests are very simple to perform, yet yield powerful results. Unlike interviews or focus 


groups, in usability tests, we rarely ask the user direct questions. Instead, we provide the user a 


task to perform on the website and then watch what they do. 


There are seven essential steps to conducting a usability test: 


1. 
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Know why you are doing the test. By following the steps in the continuous improvement plan 
above, you will identify the goal of your website, and you will understand what success will look 
like. Before you begin a usability test, make sure you know what research questions you are 
hoping to answer with the test. 


. Create tasks. Usability tests work best when you give the user a specific task to do. For example, 


if a website sells clothing, you might ask the user to select a shirt to buy and begin the 
purchasing process. 


. Create a script. The script will usually have a few standard sections. First, introduce the test to 


set the stage for your participant. Here, reassure the participant that they are not being tested 
and that there are no wrong answers. Next, ask a few questions to get a sense of the participant 
and their familiarity with this type of website. Then, show the website to the participant and ask 
for their reactions. Finally, provide them with the tasks and watch what they do. Throughout, 
ask them to speak aloud. 


. Recruit participants. Research has shown that around five testers are all you need for a 


usability test. Any more, and the information becomes repetitive. Any less, and you might miss 
something. Ideally, these testers will reflect your website users. For example, if your website 
served pet owners, then ideally your testers would also own pets. However, the testers do not 
always need to perfectly reflect your website users. Even if an individual did not own pets, they 
still might be able to pick up on navigational issues or other concerns with a website for pet 
owners. If you face a choice between doing testing with a less-than-perfect tester, or no testing, 
itis better to do the testing. 


. Conduct the tests. During the test, allow the participant to take control of the mouse (either 


through screen sharing or by physically giving them the mouse during in-person testing). Let the 
participant figure out what they should do, while you remain neutral yet encouraging. 
Acknowledge what they are saying and let them know that you would like them to continue, but 
do not directly answer questions or guide their actions. To help you focus on the participant, 
either have someone else take notes or audio-record the session with the participant’s 
permission. 
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6. Think about what you learned. After each session, jot down the top three issues you identified. 
After you have completed five testing sessions, pause to reflect on what you’ve learned. User 
testing will always uncover a host of issues, so it’s important to think about which issues are the 
most important to address. When prioritizing issues, think about why you originally set up the 
tests and what questions you were hoping to answer. Always rank your list of issues and 
recommendations by 1) which are most critical to your goals and 2) which you can do something 
about. 


7. Make fixes. These could range from minor tweaks to larger changes. Regardless, make a list of 
the things you want to change and order them by priority (as well as how much time you have 
available). 
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